Document secrets #568

mdlinville · 2016-11-15T01:22:20Z

Describe the proposed changes

Add docs for using secrets, addresses #529.

Unreleased project version

Engine 1.13

Related issue

#529

Related issue or PR in another project

see #529

Please take a look

@ehazlett @thaJeztah @mrjana @stevvooe and please add anyone else who should review.

stevvooe · 2016-11-18T21:54:43Z

_data/toc.yaml

@@ -219,6 +219,8 @@ toc:
      title: Manage nodes in a swarm
    - path: /engine/swarm/services/
      title: Deploy services to a swarm
+    - path: /engine/swarm/store_sensitive_strings/


Why not secrets?

oh, I suppose.... I just don't think secrets means anything to anyone. :)

Agreed. Let's use sensitive data.

Non-blocking: it looks like in the rest of the documentation there is a parenthetical "(secrets)" after "manage sensitive strings" - should that also be included here so folks can more easily find it in the TOC if they're specifically looking for the word "secrets"?

I just don't think secrets means anything to anyone. :)

This is literally the name of the feature. If it doesn't yet have meaning, we need to make it have meaning.

But, it is literally the industry standard word:

https://spring.io/blog/2016/06/24/managing-secrets-with-vault
http://kubernetes.io/docs/user-guide/secrets/
moby/moby#13490
http://techblog.mdsol.com/2015/10/30/Using-Secrets-Safely-in-Docker-Builds.html
https://square.github.io/keywhiz/
https://eng.lyft.com/announcing-confidant-an-open-source-secret-management-service-from-lyft-1e256fe628a3#.c72aeukpn

stevvooe · 2016-11-18T21:57:08Z

engine/swarm/store_sensitive_strings.md

+itself has already been propagated to the swarm node using mutual TLS.
+
+>**Warning**: RAFT data is encrypted in Docker 1.13 and newer. If some of your
+Swarm nodes run an earlier version, the secrets are stored unecrypted in


the secrets may be stored unecrypted in those nodes' RAFT logs.

I'm not sure that we opaquely store secret data in older versions.

@diogomonica @cyli ?

If we're in a situation where there is a 1.12 manager in a 1.13 cluster, and someone adds a secret, the 1.12 manager will write the secrets unencrypted to disk.

The upgrade path should be: upgrade all the managers to 1.13, and then start using secrets.

We should probably add something about how to securely upgrade from a 1.12, or a note that to use secrets securely, make sure that all of the manager nodes are running 1.13

@diogomonica While RAFT messages may be written to disk opaquely, I'm not sure that a manager with 1.12 snapshot structure definition will actually do this. None of our types have opaque field preservation, as far as I can tell. Looks like this was removed in proto3, as described in protocolbuffers/protobuf#272.

The upgrade path should be: upgrade all the managers to 1.13, and then start using secrets.

I agree and this language is sufficient to protect users against leaks, even if not precisely accurate. We may want to make this stronger and warn users to make sure the entire cluster is on 1.13 before performing cluster mutations.

The only data loss scenario @aaronlehmann and I could work out was if a 1.12 engine was elected leader in a mixed-version cluster. We may be able to downweight older engines in the master election process to minimize this possibility.

The other scenarios for data loss considered are 1.12 proxying to 1.13 leader, but this is a non-issue, since the JSON API prevents opaque field transfer.

The only data loss scenario @aaronlehmann and I could work out was if a 1.12 engine was elected leader in a mixed-version cluster.

Or if that engine is upgraded to 1.13 and later becomes the leader. It will be missing the data.

@diogomonica While RAFT messages may be written to disk opaquely, I'm not sure that a manager with 1.12 snapshot structure definition will actually do this. None of our types have opaque field preservation, as far as I can tell. Looks like this was removed in proto3, as described in protocolbuffers/protobuf#272.

It might be written as a WAL.
Update: tested this manually - it is not written to the snapshot, but it is written to the WAL.

stevvooe · 2016-11-18T22:00:19Z

engine/swarm/store_sensitive_strings.md

+revoke its access to a given secret at any time.
+
+The Engine managing a given worker node creates a bind mount within
+`/run/secrets/` on the worker node for each secret. From the point of view of


@ehazlett /run/secrets runs afoul of FHS standards. Do we link to /var/run to /run?

From https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard:

In FHS 3.0, /var/run is replaced by /run; a system should either continue to provide a /var/run directory, or provide a symbolic link from /var/run to /run, for backwards compatibility.[10]

I think we just need to make sure that /run is not a fork of /var/run. Accessing /var/run/secrets or /run/secrets should be equivalent, recommending the later.

Yes /var/run is linked to /run

stevvooe · 2016-11-18T22:02:44Z

A few nits, but, otherwise, a great doc!

diogomonica

Looks good overall.

We're missing examples and a description of the advanced syntax for secrets, and more importantly, an example of how to rotate a secret.

diogomonica · 2016-11-18T22:01:38Z

engine/swarm/store_sensitive_strings.md

@@ -0,0 +1,234 @@
+---
+title: Manage sensitive strings (secrets) for Docker services
+description: How to securely store, retrieve, and use sensitive strings (secrets) in Docker services


the use of strings is weird. what about sensitive data everywhere?

Could you store sensitive data that isn't a string?

String has a very specific meaning for developers and programmers. I think we want to avoid using that term. The term blob would technically be more adequate, but I think sensitive data reads a lot better.

diogomonica · 2016-11-18T22:02:07Z

engine/swarm/store_sensitive_strings.md

+
+## About secrets
+
+In terms of Docker Swarm services, a _secret_ is a string, such as a password,


string -> a blob of data

diogomonica · 2016-11-18T22:02:23Z

engine/swarm/store_sensitive_strings.md

+## About secrets
+
+In terms of Docker Swarm services, a _secret_ is a string, such as a password,
+credit card number, SSH private key, SSL certificate, or another piece of


Definitely not a credit card number. Lets remove that.

diogomonica · 2016-11-18T22:02:48Z

engine/swarm/store_sensitive_strings.md

+In terms of Docker Swarm services, a _secret_ is a string, such as a password,
+credit card number, SSH private key, SSL certificate, or another piece of
+sensitive data that should not be transmitted over a network or stored in a
+Dockerfile or in your application's source code in plain text. Starting in


starting with?

diogomonica · 2016-11-18T22:03:19Z

engine/swarm/store_sensitive_strings.md

+Docker 1.13, Docker can encrypt and manage these secrets for you. Consider the
+following examples when you might want to manage sensitive strings using Docker:
+
+- Your service relies on a password-protected database, and you want to protect


and you want to securely distribute the credentials to your application containers

diogomonica · 2016-11-18T22:12:11Z

engine/swarm/store_sensitive_strings.md

+itself has already been propagated to the swarm node using mutual TLS.
+
+>**Warning**: RAFT data is encrypted in Docker 1.13 and newer. If some of your
+Swarm nodes run an earlier version, the secrets are stored unecrypted in


We should probably add something about how to securely upgrade from a 1.12, or a note that to use secrets securely, make sure that all of the manager nodes are running 1.13

diogomonica · 2016-11-18T22:13:03Z

engine/swarm/store_sensitive_strings.md

+`/run/secrets/` on the worker node for each secret. From the point of view of
+the container, the secret is available as plain text. The `/run/secrets/`
+directory is an in-memory filesystem. When the node stops or leaves the swarm,
+or when the service is stopped or removed, the secrets are no longer accessible.


add: and are completely removed from the node's memory.

diogomonica · 2016-11-18T22:14:01Z

engine/swarm/store_sensitive_strings.md

+## Read more about `docker secret` commands
+
+Use these links to read about specific commands, or continue to the
+[example about using secrets with a service](store_sensitive_strings.md@example-use-secrets-with-a-service).


I like store sensitive data instead of sensitive strings

diogomonica · 2016-11-18T22:20:56Z

engine/swarm/store_sensitive_strings.md

+When evaluating the value of the environment variable,
+first check whether that value corresponds to a file path available within the
+container read the credentail from the file if so. The `mistysj/wordpress` image
+adds the following check to see if the value of `WORDPRESS_DB_PASSWORD` is a


We should probably document why we don't support environment variables in the first place. They are less secure than files.

See: moby/moby#9176 (comment)

diogomonica · 2016-11-18T22:22:17Z

engine/swarm/store_sensitive_strings.md

+- [`--secret`](../reference/commandline/service_create.md#create-a-service-with-secrets) flag for `docker service create`
+- [`--secret-add` and `--secret-rm`](../reference/commandline/service_update.md#adding-and-removing-secrets) flags for `docker service update`
+
+## Example: Use secrets with a service


We need an example for rotation of a secret (add a new secret, remove the old one, and ensure the new secret has the same target as the old one.

We should probably also give advanced examples on how to change the permissions of the secret file, and the use of source=bla,target=bla,uid=1000,gid=1000,mode=0700

thaJeztah · 2016-11-21T14:21:38Z

Hm, for fun I tried adding a binary as secret, and that works 😄

$ docker run --name hola hello-world

$ docker cp hola:/hello .
$ cat hello | docker secret create hello
r4gmhiw6fmiorqfk5reqbyryr

$ docker service create --name foo --secret source=hello,target=hello,mode=0755 nginx:alpine


$ docker exec $(docker ps -n1 -q --filter label=com.docker.swarm.service.name=foo) /run/secrets/hello

Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
 3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker Hub account:
 https://hub.docker.com

For more examples and ideas, visit:
 https://docs.docker.com/engine/userguide/

Should we document what size limits there are to secrets? (I don't think we put a size limit on there, but given that secrets are in memory, using many / big data there could have consequences perhaps?)

ehazlett · 2016-11-21T15:12:08Z

Yes there is a size limit of 500 KB (https://github.com/docker/swarmkit/blob/master/manager/controlapi/secret.go#L22) per secret. We should document this.

mdlinville · 2016-11-21T20:24:39Z

@diogomonica @ehazlett @stevvooe @cyli @toli PTAL. I think the topic is starting to shape up nicely. I addressed the feedback so far, and added a section on rotating secrets. Still waiting on an update to the official mysql image so that I can use that in the example instead of mysql/mysql-server.

mdlinville · 2016-11-21T21:14:29Z

cc/ @johndmulhausen @hansfrederickbrown

stevvooe · 2016-11-21T21:15:44Z

LGTM

mdlinville · 2016-11-21T21:16:45Z

Added info about binary content and 500kb limit.

cyli · 2016-11-21T21:57:05Z

engine/swarm/secrets.md

+      different target file name within the container. The WordPress container
+      will use the mount point `/run/secrets/wp_db_password`. Also specify that
+      the secret is not world-readable, by setting the mode to `0400`.
+    - Sets the environment variable `WORDPRESS_DB_PASSWORD` to this file path. The


WORDPRESS_DB_PASSWORD_FILE (correctly set in the example down below)

cyli · 2016-11-21T21:57:29Z

engine/swarm/secrets.md

+decrypted secret is mounted into the container in an in-memory filesystem at
+`/run/secrets/<secret_name>`. You can update a service to grant it access to
+additional secrets or revoke its access to a given secret at any time. When the
+task container stops running, the secret is unavailable to that container. When


Non-blocking: do you mean if you docker start the stopped container?

No, I was trying to cover the case where a node is running more than one task, each of which may have access to the same secret. When one of the containers stops, it doesn't have access to the secret anymore. If none of a node's task containers need access to a given secret anymore, I think the secret is flushed from the node as well, unless the node is acting as a backup master. Right? Is there a better way to put this?

I think your new change is great - I was wondering if you were specifically trying to call out doing a commit in a running container so it'd be available later, or restarting a container, or otherwise checking the contents of the stopped container on disk, etc.

cyli · 2016-11-21T21:58:47Z

engine/swarm/secrets.md

+      will use the mount point `/run/secrets/wp_db_password`. Also specify that
+      the secret is not world-readable, by setting the mode to `0400`.
+    - Sets the environment variable `WORDPRESS_DB_PASSWORD` to this file path. The
+      `mistysj/wordpress` image is a fork of the official WordPress image which


The example below uses the wordpress:latest image now (this comment refers to mistysj/wordpress). :) Thanks for getting the official image updated!

cyli · 2016-11-21T23:43:26Z

engine/swarm/secrets.md

+
+## Example: Rotate a secret
+
+This example builds upon the previous one. In this scenario, you create a new


"create a new secret with..."

mdlinville · 2016-11-22T00:38:50Z

Addressed @cyli feedback, tried to reword the area about when secrets get removed from a node's memory. PTAL at that section again and give me suggestions how I can improve it.

diogomonica

Again, this is looking better and better. We need

Another document that talks about lock/unlock/rotation that we link from this document when we talk about encryption
An easier example for secrets, for example (in one swarm manager cluster):

# docker secret create secret_data.txt
# docker service create --name redis --secret=secret_data.txt redis:alpine 
# docker ps
# docker exec -it redis cat /run/secrets/secret_data.txt

I think this would show exactly how this works faster than the MySQL example.

diogomonica · 2016-11-22T00:50:10Z

engine/swarm/secrets.md

+those containers that need access to it. A _secret_ is a blob of data that is
+encrypted during transit and at rest in a Docker swarm. A given secret is only
+accessible within those containers which have been granted explicit access to it,
+and only while those containers are running.


maybe we replace "accessible within those containers" to "accessible to those services"?

Other than that, beautiful.

diogomonica · 2016-11-22T00:50:26Z

engine/swarm/secrets.md

+runtime but you don't want to store in the image or in source control, such as:
+
+- Usernames and passwords
+- Database names


database credentials

I really did mean the database name, separately from the credentials (username and password). For instance, you can specify that as a file with the new wordpress image enhancements.

Reworded a bit.

diogomonica · 2016-11-22T00:54:20Z

engine/swarm/secrets.md

+
+When you add a secret to the swarm, Docker sends the secret to the swarm manager
+over a mutual TLS connection. The secret is stored in the RAFT log, which is
+encrypted. The entire RAFT log is replicated across the other managers, ensuring


We should probably link here to another document talking about unlock and unlock-key, and how the key used for encryption is stored and how it can be managed/rotated/etc.

Waiting for #694 to be merged.

diogomonica · 2016-11-22T00:55:28Z

engine/swarm/secrets.md

+secret with the new MySQL password, update the `mysql` and `wordpress` services
+to use it, then remove the old secret.
+
+1.  Create the new secret, and name it `mysql_password_2016`.


let's use mysq_password_v2 instead of 2016

diogomonica · 2016-11-22T00:56:43Z

engine/swarm/secrets.md

+## Build support for Docker Secrets into your images
+
+If you develop a container that can be deployed as a service and accepts a
+sensitive string, such as a credential, as an environment variable, consider


and requires sensitive data, such as...

diogomonica · 2016-11-22T00:58:19Z

engine/swarm/secrets.md

+being passed directly.
+
+>**Note**: Docker secrets do not set environment variables directly. This was
+a conscious decision, since environment variables can leak between containers


I would do since it's easy for environment variables to be unintentionally leaked

cyli · 2016-11-22T03:36:37Z

@mstanleyjones This looks great, thank you! I just wasn't sure if you were trying to emphasize something in particular, and this removes that confusion. Appreciate all your hard work on this! Other than @diogomonica's suggestion for lock and unlock, this all LGTM!

ehazlett · 2016-11-22T13:23:52Z

engine/swarm/secrets.md

+## Read more about `docker secret` commands
+
+Use these links to read about specific commands, or continue to the
+[example about using secrets with a service](secrets.md.md@example-use-secrets-with-a-service).


Should this be secrets.md@example-use-secrets-with-a-service?

probably @ -> # as well?

ehazlett · 2016-11-22T13:26:20Z

one minor comment otherwise LGTM -- nice work!

cyli · 2016-11-22T23:27:07Z

engine/swarm/secrets.md

+      lwxgks2rojdbvaui5kay1suvr
+    ```
+
+    The value returned is not the password, but the ID (digest) of the secret.


Oops, sorry I missed this earlier. The ID is not the digest - the ID is opaque.

cyli · 2016-11-22T23:27:44Z

engine/swarm/secrets.md

+
+    ```bash
+    $ docker secret ls
+    ID                          NAME                CREATED             UPDATED             SIZE


Also, many apologies for springing this on you at the last minute, but moby/moby#28727 (comment) removes displaying sizes and digests at all when you inspect. Could you delete the last column in this sample output? (the size column?)

mdlinville · 2016-11-23T19:06:46Z

Addressed latest feedback. Thanks @cyli !

mdlinville · 2016-11-23T19:14:16Z

Modified the instructions in the mysql lab now that docker-library/mysql#237 is merged and the mysql image has been updated!

cyli · 2016-11-23T21:38:08Z

engine/swarm/secrets.md

+    The secret is stored in the encrypted RAFT logs for the swarm.
+
+4.  Create the MySQL service. The `mysql` image now supports
+    additional environemnt variables which read their values from a file within


"environment" mispelling :)

Also, non-blocking nitpick - this reads to me like the environment variable reads values from a file? Whereas it's the script within the image which reads the file pointed at from the environment variable. I'm not sure the best way to say this in a clear way, but maybe something like "additional environment variables, which lets mysql read secret values from a file..."

That may make the next sentence a bit redundant though.

cyli · 2016-11-23T21:45:40Z

@mstanleyjones Thank you! I have a minor spelling/phrasing nitpick, but everything looks great! 👍 LGTM!

cyli · 2016-12-21T23:59:47Z

@thaJeztah Also, 👍 on the separate wordpress and mysql users!

thaJeztah · 2016-12-22T00:24:52Z

wow your shell-fu is WAY better than mine!

"Learn by example" - I looked into some of @tianon's scripts to see how it's done 👑

Also, in the revised example, should we mention that the service update to mysql causes the container to restart and thus would cause a disruption in the wordpress service?

Probably good to mention.

My goals were, first of all, to avoid having password in plain text on-disk. I know a lot of work has gone into the secrets feature to avoid just that, and ending up with writing to plain text files in the documentation didn't feel "right".

At first, I wanted to avoid the docker exec as a whole, and spin up a mysql container to use as client (which is the "neat" way to do it). Unfortunately, we don't have secrets yet on docker run, so that didn't work.

As a follow up to this PR, we should do some thinking about how to actually use the secrets in various situations. For example, the WordPress image still sets the secrets through environment variables, so a stray phpinfo() page (yup expect those to be in a lot of setups) will happily show the password;

Preventing that needs changes to the configuration files of images, so we may want some examples for that.

cyli · 2016-12-22T01:01:39Z

My goals were, first of all, to avoid having password in plain text on-disk. I know a lot of work has gone into the secrets feature to avoid just that, and ending up with writing to plain text files in the documentation didn't feel "right".

Yep, don't disagree, although in theory it's on your local machine, so less terrible. It can also be stored in a password manager such as the lastpass CLI, which would also let you pipe it into secrets, but wasn't sure if that'd be too much for an example.

At first, I wanted to avoid the docker exec as a whole, and spin up a mysql container to use as client (which is the "neat" way to do it). Unfortunately, we don't have secrets yet on docker run, so that didn't work.

Can we use a service with --restart-condition=none?

docker service create \
  --name=mysql_password_rotation \
  --network=wpnet \
  --secret=mysql_password \
  --secret=mysql_password_v2 \
  --restart-condition=none \
  mysql \
  bash -c 'mysqladmin --host=mysql --user=wordpress --password="$(< /run/secrets/mysql_password)" password "$(< /run/secrets/mysql_password_v2)"'

I'm not sure if we're starting to get into weird service esoteric stuff, now, though :)

As a follow up to this PR, we should do some thinking about how to actually use the secrets in various situations. For example, the WordPress image still sets the secrets through environment variables, so a stray phpinfo() page (yup expect those to be in a lot of setups) will happily show the password

Ah good catch. :| I didn't realize it'd be part of the process's environment - I'd only checked mysql's, which didn't seem to be. cc @tianon

thaJeztah · 2016-12-22T01:21:06Z

I'm not sure if we're starting to get into weird service esoteric stuff, now, though :)

definitely taking it too far. we can revisit once (if) secrets are available for docker run

Ah good catch. :| I didn't realize it'd be part of the process's environment - I'd only checked mysql's, which didn't seem to be.

Was already discussing with him; we could probably unset the env-vars directly after they have been consumed. Help me remind, I can try a pull request (or bake Tianon some cookies for christmas)

mdlinville · 2016-12-23T01:34:04Z

Pushed some updates. This puts back the mysql service and also uses @thaJeztah magical way to update the password in MySQL. The only thing it doesn't do yet is use a non-root user for the database. Need to figure out how to do that.

thaJeztah

Left some hints what needs to be changed to make the wordpress user work

thaJeztah · 2016-12-23T17:33:27Z

engine/swarm/secrets.md

+                       --mount type=volume,source=mydata,destination=/var/lib/mysql \
+                       --secret source=mysql_password,target=mysql_password \
+                       -e MYSQL_ROOT_PASSWORD_FILE="/run/secrets/mysql_password" \
+                      mysql:latest


You need two secrets here; one for the root password and one for the wordpress user;

docker service create \ --name mysql \ --replicas 1 \ --network=mysql_private \ --mount type=volume,source=mydata,destination=/var/lib/mysql \ --secret=mysql_root_password \ --secret=mysql_password \ -e MYSQL_ROOT_PASSWORD_FILE="/run/secrets/mysql_root_password" \ -e MYSQL_PASSWORD_FILE="/run/secrets/mysql_password" \ -e MYSQL_DATABASE=wordpress \ -e MYSQL_USER=wordpress \ mysql

here;

MYSQL_ROOT_PASSWORD_FILE sets the path to the secret for the root password

MYSQL_PASSWORD_FILE sets the path to the secret for the regular user's (wordpress) password

MYSQL_DATABASE sets the name of the database to create the first time the service is started

MYSQL_USER sets the name of the user to create the first time the service is started

Make sure to use the right secret for the right user.

thaJeztah · 2016-12-23T17:36:13Z

engine/swarm/secrets.md

+                     --publish 30000:80 \
+                     --secret source=mysql_password,target=wp_db_password,mode=0400 \
+                     -e WORDPRESS_DB_PASSWORD_FILE="/run/secrets/wp_db_password" \
+                     -e WORDPRESS_DB_HOST="mysql:3306" \


You need to add WORDPRESS_DB_USER here, to set the name to use for connecting to the MySQL database;

-e WORDPRESS_DB_USER=wordpress \

thaJeztah · 2016-12-23T17:37:00Z

engine/swarm/secrets.md

+                     --replicas 1 \
+                     --network mysql_private \
+                     --publish 30000:80 \
+                     --secret source=mysql_password,target=wp_db_password,mode=0400 \


Add a volume here, so that the WordPress data is preserved when updating the service;

--mount type=volume,source=wpdata,destination=/var/www/html \

What WordPress data is that? I thought it stored everything in the MySQL database....

The WordPress installation (theme, uploads, configuration, etc)

thaJeztah · 2016-12-23T17:42:15Z

engine/swarm/secrets.md

+    ```bash
+    $ docker service rm wordpress
+
+    $ docker rm -f mysql


this will be docker service rm mysql, or combine with the previous one (docker service rm wordpress mysql)

also, the named volumes can be cleaned up afterwards

mdlinville · 2016-12-28T00:01:13Z

OK, the latest update uses a separate wordpress database, user, and secret. I still need to set the the MySQL root password, but it's not used after MySQL starts up.

This is getting very close to being ready to go, IMHO. I guess the only outstanding concern is the WordPress environment variable, right @thaJeztah ?

mdlinville · 2016-12-28T00:03:49Z

BTW I don't need to use WORDPRESS_DB_USER because it defaults to wordpress.

thaJeztah · 2016-12-28T08:10:14Z

I guess the only outstanding concern is the WordPress environment variable

That's something to be addressed in the official WordPress image; I haven't had time to look into that yet, but it's orthogonal to this PR

BTW I don't need to use WORDPRESS_DB_USER because it defaults to wordpress.

Oh, yes, it probably defaults to that; doesn't hurt to be explicit, but I'm also fine to skip setting it

thaJeztah

Some last bits and nits, but I think we're almost there

thaJeztah · 2016-12-28T22:41:21Z

engine/swarm/secrets.md

+    l1vinzevzhj4goakjap5ya409
+    ```
+
+    The value returned is not the password, but the ID (digest) of the secret.


small nit; remove (digest) here; the ID of secrets have no relation to the content of the secrets (otherwise they may potentially "leak" the content).

thaJeztah · 2016-12-28T22:42:39Z

engine/swarm/secrets.md

+
+    - Because the scale is set to `1`, only a single MySQL task runs.
+      Load-balancing MySQL is left as an exercise to the reader and involves
+      than just scaling the service.


nit: missing "more" before "than"

thaJeztah · 2016-12-28T22:44:43Z

engine/swarm/secrets.md

+      initializing the system database for the first time. Afterward, the
+      passwords are stored in the MySQL system database itself.
+    - Sets environment variables `MYSQL_USER` and `MYSQL_DATABASE`. A new
+      database called `wordpress` is created when the image starts, and the


nit: "when the image starts" -> "when the container starts"

thaJeztah · 2016-12-28T22:47:23Z

engine/swarm/secrets.md

+    wvnh0siktqr3  mysql  replicated  1/1       mysql:latest
+    ```
+
+    At this point, you could actually revoke the `mysql` service's access to the


thaJeztah · 2016-12-28T22:52:47Z

engine/swarm/secrets.md

+Rotating passwords or other secrets will often involve additional steps outside
+of Docker.
+
+1.  Create the new password and store it in file in the `tempdir` directory


remove "and store it in file in the ..." part, as it's no longer needed

thaJeztah · 2016-12-28T22:54:31Z

engine/swarm/secrets.md

+
+    ```bash
+    $ docker service update \
+                     --secret-rm mysql_password mysql


This is something we need to fix; doing --secret-rm and --secret-add should be possible in a single update

thaJeztah · 2016-12-28T22:57:43Z

engine/swarm/secrets.md

+    First, find the ID of the `mysql` container task.
+
+    ```bash
+    $ docker ps | grep mysql |awk {'print $1'}


It's better to use the built-in filter functionality here (and -q to only show the ID;

docker ps -q --filter name=mysql

thaJeztah · 2016-12-28T23:00:58Z

engine/swarm/secrets.md

+    Verify that the blog post you wrote still exists, and if you changed any
+    configuration values, verify that they are still changed.
+
+6.  Revoke accsss to the old secret from the MySQL service and


typo accsss

mdlinville · 2016-12-29T00:40:06Z

Fixed latest problems found by @thaJeztah and also got a review from my friend @bskaggs as well. He spotted things I could no longer see, I've been looking at this too long!

thaJeztah · 2016-12-29T00:53:35Z

engine/swarm/secrets.md

+    ```bash
+    $ docker service rm wordpress mysql
+
+    $ docker volume rm mydata


need to add wpdata here as well now

thaJeztah

LGTM, left one quick nit, and possibly someone else may want to have a final look, because i'm now probably blind as well 😄

mdlinville · 2016-12-29T19:20:01Z

@thaJeztah nit fixed, and a few more picky formatting tweaks. I think this is ready to merge but I'd like input from @cyli @diogomonica @NathanMcCauley

cyli · 2017-01-03T18:44:20Z

engine/swarm/secrets.md

+    `service update` command redeploys the service.
+
+    ```bash
+    $ docker exec -it <container_id> cat /run/secrets/my_secret_data


Non-blocking: since we had the example above of docker exec $(docker ps --filter name=redis -q) cat /run/secrets/my_secret_data, can we use that same command here instead of <container_id>?

cyli · 2017-01-03T18:58:50Z

engine/swarm/secrets.md

+
+### Intermediate example: Use secrets with a Nginx service
+
+> **Note**: This example uses a single-Engine swarm for simplicity, and only


Non-blocking: Since all of these examples (redis, nginx, and wordpress) presume a single-Engine swarm, should that part of this note be moved to the top of the examples section? (Similarly for the wordpress example)

cyli · 2017-01-03T19:19:11Z

engine/swarm/secrets.md

+
+    > **Note**: Normally you would create a Dockerfile which copies the `site.conf`
+    > into place, build the image, and run a container using your custom image.
+    > This example does it all in one step.


Maybe something like "This example does not require a custom image; it copies the site.conf into place and runs the container all in one step"?

cyli · 2017-01-03T19:28:09Z

engine/swarm/secrets.md

+    yvsczlx9votfw3l0nz5rlidig   mysql_root_password   12 seconds ago      12 seconds ago
+    ```
+
+    The secrets are stored in the encrypted WAL logs for the swarm.


Non-blocking: Should we just say "Raft logs" here, since above we mention that secrets are stored in the raft logs? It's true that they are written to the WAL (and snapshot, later on), but that might be too much raft detail for this document, particularly since we never say what WAL stands for.

cyli · 2017-01-03T19:33:38Z

engine/swarm/secrets.md

+> a file on disk. You must use a query or a `mysqladmin` command to change the
+> password in MySQL.
+
+1.  Generate a random alphanumeric password for MySQL and store it as a Docker


Should we mention that this first password is for the wordpress user for mysql? Or at least a non-root user?

cyli · 2017-01-03T19:49:34Z

engine/swarm/secrets.md

+queries or commands, as opposed to just changing a single environment variable
+or a file, since the image only sets the MySQL password if the database doesn’t
+already exist, and MySQL stores the password within a MySQL database by default.
+Rotating passwords or other secrets will often involve additional steps outside


Nonblocking nitpick: should this be "may"? If it's something simple like an API key the service just access, just changing the service spec should be fine. Alternately, this is likely always the case with databases, so maybe we can call out DBs specifically here?

cyli · 2017-01-03T19:53:42Z

engine/swarm/secrets.md

+    `/run/secrets/mysql_password`.
+
+    Even though the MySQL service has access to both the old and new secrets
+    now, the MySQL root password has not yet been changed.


We don't change the root password at all in this example, right? Could we clarify this example to say that we don't need to adjust the service's access to the MySQL root password, since it's not being modified?

cyli · 2017-01-03T19:54:17Z

engine/swarm/secrets.md

+    Even though the MySQL service has access to both the old and new secrets
+    now, the MySQL root password has not yet been changed.
+
+3.  Now, change the MySQL password for the `wordpress` user using the `mysql`


"mysqladmin CLI", as opposed to "mysql CLI"

Fixes #529 Signed-off-by: Misty Stanley-Jones <[email protected]>

mdlinville · 2017-01-04T19:50:15Z

Addressed @cyli feedback, and also pushed a second commit which updates the syntax of the -f flag to docker create, as per moby/moby#29889.

cyli · 2017-01-05T23:05:32Z

LGTM!

thaJeztah

still LGTM

we can update the syntax once moby/moby#29955 was decided on.

Thanks again, @mstanleyjones !

mdlinville added this to the engine/1.13.0 milestone Nov 15, 2016

mdlinville changed the title ~~529 secrets~~ Document secrets Nov 15, 2016

mdlinville changed the base branch from master to vnext-engine November 15, 2016 01:24

stevvooe reviewed Nov 18, 2016

View reviewed changes

diogomonica suggested changes Nov 18, 2016

View reviewed changes

cyli reviewed Nov 21, 2016

View reviewed changes

diogomonica suggested changes Nov 22, 2016

View reviewed changes

ehazlett reviewed Nov 22, 2016

View reviewed changes

mdlinville mentioned this pull request Nov 22, 2016

Document locking and unlocking of swarm managers #689

Closed

cyli reviewed Nov 22, 2016

View reviewed changes

cyli reviewed Nov 23, 2016

View reviewed changes

thaJeztah requested changes Dec 23, 2016

View reviewed changes

sylnsr approved these changes Dec 28, 2016

View reviewed changes

thaJeztah requested changes Dec 28, 2016

View reviewed changes

thaJeztah reviewed Dec 29, 2016

View reviewed changes

thaJeztah approved these changes Dec 29, 2016

View reviewed changes

cyli reviewed Jan 3, 2017

View reviewed changes

Misty Stanley-Jones added 2 commits January 4, 2017 11:37

Document Docker secrets

109a564

Fixes #529 Signed-off-by: Misty Stanley-Jones <[email protected]>

Update syntax of the -f flag in the examples

c7d2879

thaJeztah approved these changes Jan 9, 2017

View reviewed changes

thaJeztah merged commit 2c26e6a into docker:vnext-engine Jan 9, 2017

mdlinville mentioned this pull request Jan 9, 2017

Document managing sensitive strings (secrets) #529

Closed


		## About secrets

		In terms of Docker Swarm services, a _secret_ is a string, such as a password,


		## Example: Rotate a secret

		This example builds upon the previous one. In this scenario, you create a new


		### Intermediate example: Use secrets with a Nginx service

		> Note: This example uses a single-Engine swarm for simplicity, and only

Document secrets #568

Document secrets #568

Conversation

mdlinville commented Nov 15, 2016

Describe the proposed changes

Unreleased project version

Related issue

Related issue or PR in another project

Please take a look

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stevvooe Nov 18, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stevvooe commented Nov 18, 2016

diogomonica left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

thaJeztah commented Nov 21, 2016

ehazlett commented Nov 21, 2016

mdlinville commented Nov 21, 2016

mdlinville commented Nov 21, 2016

stevvooe commented Nov 21, 2016

mdlinville commented Nov 21, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cyli Nov 21, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mdlinville commented Nov 22, 2016

diogomonica left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cyli commented Nov 22, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ehazlett commented Nov 22, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mdlinville commented Nov 23, 2016

mdlinville commented Nov 23, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cyli commented Nov 23, 2016 • edited Loading

cyli commented Dec 21, 2016

thaJeztah commented Dec 22, 2016

stevvooe Nov 18, 2016 •

edited

Loading

cyli Nov 21, 2016 •

edited

Loading

cyli commented Nov 23, 2016 •

edited

Loading